Content removal bias in web scraped data: A solution applied to real estate ads

Abstract I propose a solution to content removal bias in statistics from web scraped data. Content occurs when data is removed the before scraper able collect it. The based on inverse probability weights, derived parameters of survival function with complex forms censoring. apply this calculation proportion newly built dwellings Luxembourg, and run counterfactual experiment Montecarlo simulation confirm findings. results show that extent relatively small if scraping frequently compared online permanence data; it grows larger less frequent scraping.